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Abstract: 

This  report  describes  an  approximation  for  the  central 
processor  (CPU)  utilization  in  a  multiprogramming  computer 
system.   The  approximation  is  based  on  use  of  the  mathematical 
theory  of  diffusion;  its  adequancy  is  checked  numerically  and 
found  to  be  good. 


INTRODUCTION. 

Several  authors  have  recognized  that  interesting  characteristics 
of  multiprogramming  computer  systems  may  be  obtained  by  analysis  of 
the  cyclic  queues  occurring  at  CPU  (Central  Processor  Unit)  and  DTU 
(Data  Transmission  Unit);  see  Gaver  [3],  Lewis  and  Shedler  [5], 
Shedler  [8],  [9],  and  others.   A  system  feature  that  complicates  the 
probability  analysis  is  the  non-Markovian  nature  of  DTU  service: 
while  it  may  be  roughly  plausible  to  assume  CPU  service  times  (e.g., 
times  to  page  fault)  to  be  independently  and  exponentially  distributed, 
it  is  apparently  much  less  reasonable  to  make  such  a  distributional 
assumption  about  DTU  service  times.   Although  straightforward  attacks 
on  this  problem  have  been  successfully  conducted,  see  [5]  and  [8],  it 
may  be  useful  to  attempt  a  simpler,  approximate,  approach,  and  this 
is  the  objective  of  the  present  paper.   The  results  obtained  here  have 
the  virtue  of  a  refreshing  mathematical  simplicity,  but  are  of  course 
only  in  approximate  agreement  with  results  such  as  those  obtained  in 
[8].   When  we  reflect  that  our  present  models  for  multiprogramming 
are  quite  simplistic  in  the  light  of  current  knowledge  of   a)  program 
behavior  at  the  CPU,   b)  information  accessing  at  the  DTU,  and   c) 
representation  of  storage  hierarchies,  approximations  of  this  type  may 
be  quite  adequate  for  providing  insights.   We  envision  using  results 
of  this  kind  to  design  computer  systems  with  acceptable  cost-perform- 
ance characteristics.   Such  results  are  useful,  for  example,  in 
suggesting  performance  characteristics  of  devices  which  will  result 
in  a  balanced  system.   In  addition,  by  taking  cost  characteristics 


into  account,  one  can  begin  to  evaluate  technologies  from  a  total 
system  point  of  view,  and  thus  be  guided  to  select  appropriate  devices , 
With  respect  to  such  questions  we  can  consider  utilizing  our  results 
to  obtain  an  initial  idea  of  the  region  of  parameter  space  to  explore 
more  extensively  by  means  of  simulation  or  by  improved  analytical 
approximations,  several  of  which  are  currently  under  development. 


2.   THE  DIFFUSION  APPROXIMATION. 

We  consider  the  cyclic  queueing  system  (see  Figure  1)  which 
consists  of  two  sequential  stages.   The  system  is  assumed  to  serve  a 
constant  number,   J,   of  programs   (J  ^  2)  ,   each  of  which  goes  through 
both  stages  in  sequence  and  then  returns  to  the  first  stage,  this  pro- 
cess being  repeated  continuously.   It  is  assumed  that  after  completion 
of  CPU  service  a  program  moves  instantaneously  from  stage  1  to  the  tail 
of  the  queue  in  stage  2,  and  after  DTU  service  at  that  stage,  back  to 
the  tail  of  the  queue  in  stage  1.   We  shall  suppose  in  addition  that 
the  queue  in  front  of  the  CPU  and  the  queue  in  front  of  the  DTU  are 
served  according  to  a  First-In-First-Out  (FIFO)  discipline.   The  assump- 
tion that  J   is  a  constant  is  an  approximation  that  is  justified  by 
the  common  practice  of  operating  such  a  system  in  a  saturated  mode. 

In  addition,  we  assume  that  the  service  times  at  the  CPU  and 
the  DTU  are  mutually  independent,  and  that  CPU  service  times  and  DTU 
service  times,  respectively,  are  identically  distributed. 

We  attempt  here  to  give  a  brief  intuitive  account  of  our  approxi- 
mation.  Let  N  (t)   denote  the  number  of  programs  present  at  the  CPU 
at  time   t;   this  includes  those  queued  in  addition  to  the  program 
currently  in  service.   Then  if  N  (0)  =  0, 

Nc(t)  =  A(t)  -  D(t)  (2.1) 

where  A(t)   represents  the  number  of  arrivals  at  the  CPU  in   (0,t), 
and  D(t)   is  the  number  CPU  departures  in   (0,t).   If  we  neglect 
boundary  effects  at   0  and  J,   A(t)   and  D(t)   are  independent 
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renewal  processes,  so  as   t  becomes  large  A(t)   and  D(t)   are 
approximately  normally  distributed  with  means    r  .   and 


E[D]       E[S] 

Var[D]  Var[S]  .   ,       „ 

and  variances   t  ,  ■,  ,  >  3   and   t  ,   '   3   respectively;  see  Cox  [1], 

p.  40.   It  follows  that  N  (t)   is  approximately  normally  distributed 


with  mean  ut  = 


t  and  variance  a2t  = 


Var[D]    Var[S] 

(eWF    TeTsTF 


_E[D]    E[S] 
We  now  approximate  by  replacing  the  difference  of  renewal  processes 

by  a  diffusion  (Wiener  process)  with  drift  y   and  infinitesimal 

variance  a2.   Thus,   F(x,t),   the  distribution  of  N  (t)  ,   satisfies 

the  diffusion  equation 

8F       3F   a2   32F  ,„  „. 

—  =  -  p  7-+  7-  -t~y  (2.2) 

dt        dx    2   9x^ 

again  approximately;  cf.  Newell  [7],  pp.  105-107.   A  reflecting 
boundary  condition  at   x  =  0  must  be  imposed,  for  N  (t)  ^  0,   and 
another  such  boundary  condition  at  x  =  J   constrains  N  (t)   to  be 
£  J.   We  require  the  solution  to  (2.3),  subject  to  an  initial  distri- 
bution, e.g. 

/  1,    x  ;>  xQ  >  0 
F(x,0)  =  \  (2.3) 

0,    x  <  xQ, 

and  boundary  conditions 

F(0+,t)  ^  0,        F(J,t)  =  1.  (2.4) 

For  further  details  of  the  behavior  of  this  approximation  when  J  is 
infinite  see  Gaver  [4].  For  a  closed  form  solution  to  (2.3)  when  J 
is  infinite,  see  Newell  [7]. 


THE  STATIONARY  DISTRIBUTION  IN  THE  DIFFUSION  APPROXIMATION. 

The  stationary  or  long-run  distribution  associated  with  our 

problem  is   F(x)  =  lim  F(x,t)   and  satisfies 

t-x» 


g±     dzF 
2   dx^ 


dF   n 


for  x  >  0.   Routine  integrations  lead  to  the  solution 


F(x)  =  A 


1  -  B  e 


2y  -i 
~~z  x 


(3.1) 


Invocation  of  the  upper  boundary  condition  provides  that 


1  =  A 


2y 


B  e 


(3.2) 


According  to  (3.1)  and  (3.2)  we  have 


F(x)  = 


1  -  B  e 


4x 


2y  T 


(3.3) 


1  -  B  e 

It  remains  to  determine  the  constant  B.   We  suggest  several  alterna- 
tives, and  present  numerical  properties  of  these. 

(1) .   Exact  fit  for  J  =  1. 

If  only  one  program  occupies  the  system  then  renewal  theory 
provides  that  (assuming  E[D]  =  1,   as  we  shall  throughout) 


F(0+)  - 


1  +  E[S]  * 


(3.4) 


If  we  set  J  =  1,   x  =  0   in  (3.3)  and  equate  the  result  to  (3.4)  the 
value  of  B   is  determined: 


B-    E[S1   , 

a2" 
E[S]  -  e° 


(2)   Exact  fit  for  J  =  2. 

If  CPU  service  times  are  independently  distributed  copies  of 
a  random  variable  S,   and  likewise  DTU  service  times  are  independent 
copies  of  the  random  variable  D,   then  the  long-run  CPU  utilization 
U,   when  J  =  2,   given  by 

"°1-F(0+)  °  e[m1,d)1  <3-5> 

This  was  pointed  out  by  J.  Gecsei  (private  communication).   This  can 

be  seen  as  follows.   Consider  the  sequence  of  times   {t  }   at  which 

K 

either  i)  the  CPU  is  idle,  a  DTU  service  has  just  been  completed  and 
the  served  program  has  moved  to  the  CPU  stage  queue  or  ii)  the  DTU 
is  idle,  a  CPU  service  has  just  been  completed  and  the  served  program 

has  moved  to  the  DTU  stage  queue.   These  {x  }   are  regeneration  points 

K 

in  the  process,  the  mean  regeneration  time  being  E[Max(S,D)].   Since 
the  mean  amount  of  time  that  the  CPU  is  busy  between  regeneration 
points  is   E(S) ,   (3.5)  follows  from  a  basic  limit  theorem  for  regener- 
ative processes;  see  Feller  [2],  p.  365.   Making  use  of  (3.3)  we  obtain 

B-  ^ r-  •       (3.6) 


E[max(S,D) ] 


1  -  e 


4y 
a 


+   E[S]e 


4u  * 
"a' 


(3)   Exact  fit  for  large  J   (J-*>°)  . 

Of  most  practical  interest  is  the  case  of  large  J.   The 
diffusion  approximation  may  also  be  expected  to  work  best  when  bound- 
aries are  visited  infrequently,  and  this  implies  large  J  and  near 
equality  of   E[S]   and  E[D], 

In  practice  y  <  0,   so  we  may  allow  J  -*■  •  in  (3.3)  to 
discover  that 

|£x  2y 

F(x)  =  lim ^r =  1  -  B  e°  (3.7) 

i-B  <r 

and  hence  for  J  =  °°, 

F(0+)  =  1  -  B  (3.8) 

But  for  arbitrary  queueing  systems  of  the  type  under  consideration 
(see  Takacs  [10]  ,  p.  142), 


lim  P{N_(t)  =0}=l-p,   if  p  <  1 


where  p   is  the  traffic  intensity  parameter: 


E[S1 


when  J  =  °°.   Consequently  we  put  B  =  p   and  find 


F(0+)  =  ^ 1—  (3.9) 

^  J 
1  -  p  e 


This  simple  expression  promises  to  give  a  good  approximation  when  J 
is  reasonably  large  and   u   is  not  far  from  zero  but  has  a  negative 
sign.   The  quality  of  the  approximations  may  be  judged  by  making 
reference  to  the  numerical  examples  that  follow. 
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4.   NUMERICAL  EXAMPLES. 

In  [8],  Shedler  has  tabulated  CPU  utilization,  namely  the  long- 
run  probability  that  the  CPU  is  busy;  the  latter  depends  upon  the  DTU 
distribution  and  the  degree  of  multiprogramming,   J.   The  numbers 
obtained  are  the  result  of  a  semi-Markov  analysis;  the  success  of  the 
latter  seems  to  depend  rather  crucially  upon  the  assumption  of  an 
exponential  service  time  at  the  CPU. 

Our  diffusion  approximation  is  capable  of  supplying  figures  for 
CPU  utilization — an  explicit  formula  follows  directly  from  (3.9)  by 
subtraction  from  unity: 

2^  T 

CPU  utilization  =  p  J- — ^ \  (4-D 

V         0~^~    ' 

1  -  p  e 
when  p  <  1.   Here  we  compare  the  "exact"  numbers  obtained  in  [8]  with 
those  delivered  by  (4.1).   Since  the  usual  system  is  likely  to  have 
a  relatively  high  degree  of  multiprogramming,  we  shall  first  discuss 
Method  3  for  fitting  B   (exact  fit  for  J  =  ») .   Refer  to  Table  1. 
Evidently  the  Diffusion  (Diff)  and  semi-Markov   (S-M)   figures  agree 
quite  closely.   The  numerical  comparisons  suggest  that  the  diffusion 
approach  provides  an  underestimate  in  the  case  of  an  exponential  DTU. 
That  this  will  always  be  the  case  may  be  shown  analytically.   Before 
doing  so,  we  shall  compare  some  of  the  results  of  Table  1  to  the  num- 
bers obtained  when  Method  2  is  used  (exact  fit  for  J  =  2) .   This 
comparison  is  illustrated  in  Table  2.   Apparently  the  procedure  of 
fitting  J  =  2   exactly  does  not  agree  as  well  with  the  S-M  calcula- 
tions as  does  the  J  =  °°  exact  fit.   When  we  recollect  that  explicit 
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calculation  of   E[max(S,D)]   for  general  distribution  of   S   and   D 
is  a  bit  troublesome,  it  becomes  easier  to  recommend  the  method  of 
fit  obtained  by  Method  3. 
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5.   THE  DIFFUSION  APPROXIMATION  TO  THE  MARKOVIAN  CYCLIC  SYSTEM. 

In  this  section  we  shall  show  that  the  tendency  for  the 
diffusion  (in  the  fit  of  Method  3)  to  underestimate  CPU  utilization 
in  the  exponential  CPU  and  DTU  (entirely  Markov)  case  is  no  accident 

Denote  the  mean  of   S,   the  exponential  service  time  at  the 
CPU,  by  E[S]  =  X~  ;   again    E[D]  =  1.   Then  explicit  probability 


balance  equations  may  be  written: 


X  P0  =  Pl 


(X+l)p.  ■  X  p .  -  +  p....     1  £  i  £  J  -  1 
i      l-l    l+l 


PJ  =  X  PJ-1, 

where  p.   is  the  long-run  probability  of   j   programs  at  the  DTU, 
The  solution  is,  expressed  in  terms  of  p  =  A   , 


1  -  p   nJ-i 


Pi  =  ,    J+l  P 


Thus 


CPU  utilization  =  U  =  1  -  pT 

e        J 

where  the  subscript  signifies  "exact;"  the  corresponding  figure  for 
the  diffusion  is  U  ,   and  the  figures  for  idleness  are  I  ■  p  ,   and 
I ,  =  1  -  U  ,   respectively.   It  will  be  shown  that 

d    e 
or 
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1  -  P 


1  ~  P 


1  -  p  e 


2^  T    1    nJ+1 

-jr  J   1  -  p 


Now  in  the  present  case  — £-  =  2  (— — -J  ,   and  an  easily  verified  inequality 
(2,n  x  <  2(^—-)   for   0  <  x  <  1)   shows  that  p  <  exp  2  p— -) 


which 


verifies  the  assertion.   It  may  also  be  shown  that  the  error  committed 
by  use  of  the  diffusion  approximation  decreases  as  J   increases,  but 
not  to  zero.   The  approximation  improves,  however,  as   E[S]  ■+  E[D]. 
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6.   DIFFUSION  APPROXIMATION  FOR  AN  EXTENSION  OF  THE  MODEL. 

In  this  section  we  shall  indicate  how  a  diffusion  approximation 
for  CPU  utilization  can  be  obtained  when  the  assumption  of  identically 
distributed  CPU  service  times  is  relaxed  to  permit  the  J   programs 
to  have  different  page-fault  characteristics.   Specifically  we  shall 
now  assume  that  the  sequence  of  CPU  service  times  are  mutually  inde- 
pendent, and  that  the  CPU  service  times  of  program  i  are  identically 
distributed  as  a  random  variable  S  .  (l£l£j)  .   The  diffusion  approxima- 
tion proposed  will  again  be  of  the  form  (4.1),  and  we  seek  appropriate 
expressions  for  u,   a2  and  p. 

It  is  easily  verified  that  the  successive  CPU  service  times  S 
in  the  cyclic  queueing  system  are  a  semi-Markov  process,  and  that 
within  this  semi-Markov  process  Z(t),   the  total  number  of  page-faults 
in  time  t,   is  a  cumulative  stochastic  process ;  see  Cox  [1] .   It 
then  follows  from  cumulative  process  theory  that 


.   ..      E{Z(t)}       J ,,    .x 

Uz   =   lim— I -  E[S7]   +    ...   +E[SJ  U'U 

L  X  J 

2  „      Var{Z(t)}        j2^h]   +    ...+Var[Sj]) 

°   z        t™  t  "  (E[S^]   +    ...   +  E(Sj))J 


(Var[S    ]   +    ...   +  VarfSj]) 

J 

rE[S1]   +    ...   +  EtSjh3 


(6.2) 


To  establish  the  drift  and  infinitesimal  variance  for  the  diffusion 
approximation  put 
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y  = 


1    Mz  "  X    E[S_]  +  ...  +  E[S1  * 


(6.3) 


and 


a2  = 


Var[D]     2   _  Var[D] 
E[D]J    a  z  "  (E[D])^  + 


P    = 


x         E[S]_]    +    ...   +  E[Sa] 


Var[S    ]   +   ...  +  Var[S    ]i 


,[S1]    +    ...    +  E[SjhJ 


(6.4) 


(6.5) 


One  can  also  assume  that  each  program's  DTU  service  time  comes  from 
a  specific  distribution,  and  generalize  the  above  expressions 
accordingly  in  an  obvious  way. 
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7.   CONCLUSIONS  AND  DIRECTIONS  FOR  FURTHER  WORK. 

Although  certain  of  the  cyclic  queue  problems  relevant  to 
multiprogramming  studies  may  be  solved  by  conventional  methods  (semi- 
Markov  processes,  or  the  "phase"  approach  described  in  Morse  [6]), 
the  results  are  nearly  always  cumbersome  and  difficult  to  compute. 
The  approximate  approach  exhibited  here  furnishes  simple  formulas 
that  may  be  evaluated  by  hand  computation.   The  accuracy  of  the  approxi- 
mation seems  quite  adequate  for  the  cases  studied.   However,  efforts 
to  improve  the  approximation  are  suggested  for  the  following  reasons. 

(1)  Data  analysis  of  actual  CPU  service  times  indicate  that  the 
latter  are  apt  to  be  somewhat  more  skewed  than  the  exponential. 

(2)  The  cyclic  queue  model  discussed  here  may  also  be  used  to  repre- 
sent a  reliability  situation.   In  this  interpretation  components 
of  subsystems  take  the  place  of  programs.   A  DTU  service  time 
represents  the  service  life  of  a  component,  and  CPU  service  is 
identified  with  a  repair;  in  this  model  there  is  only  one  repair 
facility.   Finally,  J  -  1  is  the  number  of  spare  components  in 
the  system.   One  question  of  interest  concerns  the  long-run  proba- 
bility that  the  DTU  is  "busy,"  equivalent  to  the  probability  that 
all  J   items  in  the  system  are  not  undergoing  repair.   This 
"availability"  figure  is  approximated  by  F(J-l).  Another  question 
relates  to  the  probability  that  if  initially  (i)  a  new  component 

is  in  operation  (at  the  DTU) ,  and  (ii)  the  J  -  1  spares  are 
queued  behind  the  latter,  then  the  time  until  the  DTU  becomes 
idle  for  the  first  time  exceeds   t.   The  latter  probability  may 
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be  approximated  by  solving  a  diffusion  equation,  this  time  with 
one  reflecting  and  one  absorbing  barrier.   The  details  remain  to 
be  worked  out,  and  the  quality  of  the  approximation  evaluated. 
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DTU  Service  Distribution    (Mean  =   1) 


Erlang  -   1  Erlang  -  «> 

(exponential)   Erlang   -   2  Erlanq  -  3     Erlang  -  4     Erlanq  -  5     (constant) 

f.[S]     S-M         Diff.    S-M       Diff.    S-M     Diff.      S-M     Diff.     S-M     Diff.     S-M     Diff. 
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3 
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.706 

.709 

.710 

.711 

.712 

.723 

.722 

5 

.696 

.695 

.716 

.716 

.722 

.722 

.726 

.725 

.728 

.727 

.736 

.734 

6 

.712 

.711 

.728 

.727 

.733 

.732 

.736 

.734 

.737 

.736 

.743 

.741 

2 

.590 

.589 

.614 

.635 

.624 

.652 

.630 

.661 

.633 

.666 

.651 

.688 

3 

.661 

.661 

.689 

.697 

.700 

.710 

.701 

.717 

.709 

.721 

.726 

.737 

4     .80 

.702 

.702 

.728 

.732 

.738 

.742 

.743 

.747 

.746 

.750 

.759 

.762 

5 

.729 

.729 

.751 

.753 

.759 

.761 

.763 

.764 

.766 

.767 

.776 

.776 

6 

.747 

.747 

.766 

.766 

.773 

.773 

.776 

.776 

.778 

.778 

.786 

.784 

2 

.631 

.631 

.656 

.685 

.667 

.705 

.673 

.715 

.677 

.721 

.694 

.748 

3 

.709 

.709 

.739 

.753 

.751 

.769 

.757 

.777 

.762 

.782 

.780 

.803 

4     ,90 

.756 

.756 

.785 

.792 

.796 

.806 

.802 

.812 

.806 

.816 

.821 

.832 

5 

.787 

.787 

.813 

.818 

.823 

.829 

.828 

.834 

.831 

.837 

.845 

.851 

6 

.808 

.808 

.832 

.835 

.841 

.844 

.846 

.849 

.848 

.852 

.860 

.863 

Table  1.     CPU  Utilization  Comparison 
Based  on  fitting  B  for  J=«. 


(CPU  exponential  mean  E[S*J) 
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DT1J  Service  Distribution   (Mean  =   1) 


J 

E[S] 

S-M 

Exponent!' 
Diff. 
(Fit  J=2) 

al 

Diff. 
(Fit  J«o) 

S-M 

Constant 
Diff. 
(Fit  J=2) 

Diff. 
(Fit  J=~) 

2 

.233 

.233 

.233 

.249 

.249 

.241 

3 

.247 

.251 

.245 

.250 

.256 

.248 

4 

.25 

.249 

.254 

.249 

.250 

.258 

.250 

5 

.250 

.255 

.250 

.250 

.258 

.250 

6 

.  250 

.256 

.250 

.250 

.259 

.250 

2 

.429 

.429 

.424 

.463 

.468 

.464 

3 

.467 

.468 

.464 

.494 

.492 

.487 

4 

.50 

.484 

.487 

.482 

.499 

.500 

.495 

5 

.492 

.495 

.491 

.500 

.503 

.498 

6 

.496 

.500 

.495 

.500 

.504 

.499 

2 

.568 

.568 

.566 

.626 

.626 

.655 

3 

.634 

.634 

.633 

.694 

.673 

.700 

4 

.75 

.672 

.673 

.672 

.723 

.696 

.722 

5 

.696 

.696 

.695 

.736 

.708 

.734 

6 

.712 

.712 

.711 

.743 

.716 

.741 

2 

.590 

.590 

.589 

.651 

.651 

.688 

3 

.661 

.661 

.661 

.726 

.702 

.737 

4 

.80 

.702 

.703 

.702 

.759 

.730 

.762 

5 

.729 

.729 

.729 

.776 

.745 

.776 

6 

.747 

.747 

.747 

.786 

.755 

.784 

2 

.631 

.631 

.631 

.694 

.694 

.748 

3 

.709 

.709 

.709 

.780 

.757 

.803 

4 

.90 

.756 

.756 

.756 

.821 

.791 

.832 

5 

.  787 

.787 

.787 

.845 

.813 

.851 

6 

.808 

.808 

.  808 

.860 

.828  ■ 

.863 

'able  2.     CPU  Utilization  Comparison 


(CPU  exponential   mean   E[S}) 
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